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ABSTRACT 



This study evaluated the degree and quality of program 
implementation and educational outcomes associated with Kentucky's Family 
Resource and Youth Service Center, a school-based family resource and support 
program which emphasized increased involvement of families of special needs 
students and the community with their schools. Twenty centers serving 
approximately 2,000 students and their families were visited over a 2-year 
period. Factor analysis identified five factors that accounted for 88,8 
percent of the variance in implementation and outcomes. They were: (1) 

teachers' knowledge/familiarity with the Center; (2) teachers' active 
involvement with the Center; (3) family and community involvement with the 
Center; (4) the Center program's mission focus; and (5) teachers' awareness 
of Center's mission. In general, teachers' active involvement with the Center 
was the most consistent positive predictor of educational outcomes. However, 
family and community involvement with Centers was inversely related to 
educational outcomes, a finding which suggests that putting effort into 
enhancing community supports and family connection with them is a long-term 
strategy that may not show results as immediately as involvement with the 
school . (DB) 
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Introduction 

School-based family resource and support programs assume 
that socially disintegrative factors that affect families attenuate 
the ability of children to fully profit from their educational 
experiences. The primary focus of these programs is to enhance 
the participation of families in the educational process and to 
strengthen the capacity of families to enable children’s readiness 
for learning. Broadly, this is to be a^chieved by empowering 
families to access a variety of services and resources, and to 
forge cooperative links among families, schools, and communities. 
The purpose of this study was to evaluate the degree and quality 
of program implementation and educational outcomes associ- 
ated with program participation of a statewide school-based 
family support initiative. 

Method 

Program 

Kentucky's Family Resource and Youth Service Center 
program (FRYSC) Is an essential component of the Kentucky 
Education Reform Act (KERA). Modal staffing of centers consists 
of a coordinator, an assistant, and various adult and student 
volunteers. Consistent with other family support programs, their 
general mission Is to reach out and establish connections with 
families and students, schools, and community agencies; and to 
establish links or bridges among them, particularly to Increase 
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the involvement of families and the community 
with schools. They are supposed to carry out their 
tasks by adapting their services and approaches to 
local characteristics and resources; to, whenever 
possible, broker or mobilize community resources, 
rather than provide direct services; and, to empower 
families to identify and utilize formal and informal 
resources to meet their own needs. Family Resource 
Centers (FRC) serving elementary schools and 
Youth Service Centers (YSC) serving secondary 
schools proliferated rapidly to 560 centers in the 
1995‘96 school year. 

Measurement 

Qualitative and quantitative methods were used 
to evaluate these programs. The qualitative methods 
consisted of half-day site visits to centers that in- 
cluded interviews of coordinators, staff, principals, 
and parents; observation of center operations; and 
review of center workplans and other materials. 
Quantitative methods included teacher surveys, and 
a computerized management information system 
that permitted coordinators to collect data on who 
was served, what services were provided, and what 
outcomes were associated with participation. 
Outcomes used in this study were teacher pre-post 
ratings of the performance of student program 
participants on 18 educational variables, as well as 
student progress on KERA levels of academic 
proficiency. 

The goal of the qualitative evaluation was to 
“open up the black box” and specify program 
operation variables. Initial impressions of patterns 
and domains of center programmatic activities 
evolved into the development of Innovation Com- 
ponents Configuration (ICC) maps (Hall & Hord, 
1987) with which we were able to reliably summarize 
domains of program implementation and levels of 
implementation in each domain. The domains identi- 
fied were Needs Assessment. Relationship With School 
Relationship With Community, Relationship With Families, 



Advisory Council Development, Mission (improving 
students’ preparedness to profit from their educa- 
tional experience), Connectedness, and Evaluation. 
The first and last domains were not used in this 
study. Table 1 portrays the Relationship With 
School domain. By dividing the number of points a 
center received in a given domain by the total 
number of possible points in that domain, a domain 
score was derived. Center implementation could be 
profiled for each domain and an overall implemen- 
tation score across domains was also derived for 
each center. (The complete ICC map and teacher 
survey are available from the first author). 

Subjects/Sites 

Twenty centers were visited over a two year 
period. The centers served approximately 2000 
students and their families during this time. This 
number includes only students and families who 
were registered as receiving targeted services and 
does not include families or children who attended 
one-time programs such as Fall back to school 
programs, or received one-time referrals. The number 
of centers and students varied for different analyses 
as indicated in Table 2. 

Analyses 

A factor analysis conducted on the 6 implemen- 
tation domain scores and 1 1 teacher survey items for 
each center yielded five factors that accounted for 
88.8% of the variance in the 17 constituent variables. 
The factors are: 1) teacher knowledge/familiarity 
with center, 2) teacher active involvement with 
center, 3) center family and community involvement, 
4) program mission focus, and, 5) teacher awareness 
of center mission. These served as our independent 
variables in prediction equations of the relationship 
of program implementation to the dependent 
variables, the 18 educational outcomes. For each of 
the prediction equations, these latent factors were 
further condensed into an Overall Implementation 
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Table 1 

Center Relationship With School 



1. Connectedness 

1. Negative (turf/hostile). 

2 . Uninformed (inappropriate or no requests /referrals) 

3. Customers (school personnel make appropriate requests/referrals; starting to be impressed with center 
capabilities). 

4. Advocate (enthusiastic about center; see it as needed resource/capability of school). 

5. Team (school works collaboratively with center; buys into the family involvement education mission). 

2. Principal support 

1. Negative (either micromanaging or turf/hostile). 

2. Laissez-faire 

3. Supportive/customer 

4. Advocate 

5. Involved/collaborative 

3. Principal’s attitude toward family involvement 

1 Not open to it 

2 Neutral 

3 Appreciates outreach to families 

4. Promotes family involvement 

5. Has track record and strategies for family involvement. 

4. Teacher acceptance of parent involvement 

1. None 

2. Some (very little) 

3. Many (somewhat) 

4. Most /all (very much) 

5. Teacher interaction with center 

1. Refer to center: (3) all, (2) most, (1) some, (0) none, (multiply “1” by (#). 

2. Drop by center: (3) all, (2) most, (1) some, (0) none, (multiply “2” by (#). 

3. Participate in center programs/activities: (3), (2), (1), (0). (multiply "3” by (#). 

4. Assist in center programs /activities: (3), (2), (1), (0). (multiply “4” by (#). 

6. Perceived effects on school performance 

1. Center is not seen as improving student attendance, classroom performance, or student achievement. 

2. School personnel are unsure about the effect of the center on student attendance, classroom performance, or 
student achievement. 

3. Center seen as important, but school personnel unsure of impact. 

4. Center is clearly seen as improving student attendance, classroom performance, and achievement. 
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variable. The relative contribution of each of these 
factors varied somewhat as a function of the specific 
educational outcome being predicted. Educational 
outcomes were computed as a “success rate” for each 
center, which was a continuous variable consisting of 
percent of students served by a center who improved 
on a given educational variable, either by changing 
yes-no categories in the desired direction or by 
moving to a higher level in a category. We then 
applied each of these variables to the data for indi- 
viduals, grouped by center, and weighted by the 
number of individuals served by the 
center. Finally, we assessed the 
relationship between Overall Imple- 
mentation and changes in student 
level of academic proficiency on the 
KERA tests. For this correlation, we 
restricted our sample to students 
who, at intake, were in one of the two 
lower KERA categories of Novice and 
Apprentice (accounting for about 80% 
of the cases) and assessed who moved 
to either Proficient or Distinguished 
categories. 

Results 

The multiple correlations 
between Overall Implementation and 
educational outcomes are presented 
in Table 2. For those variables that 
are stated in negative terms (e.g. risk, 
tardiness), the correlation is in a 
negative direction, as would be 
predicted. The achievement variable 
(at/below grade level) is coded 
inversely, so the correlation is 
positive. That is, the greater the 
implementation, the better the gains 
with respect to achievement. All 
correlations are significant at the 
p< .01 level. Clearly, the extent to 



which centers have implemented their programs is 
positively related to educational improvement 
among participating students. 

The results of the regression equations for each 
the education variables with the contribution of each 
of the five factors expressed as beta weights are 
presented in Table 3. Beta weights are standardized 
regression coefficients: a measure of how much of a 
change in standard deviations in the dependent 
variable is caused by one standard deviation shift in 

Table 2 

Correlation Between Overall Implementation 
and Educational Outcomes 



Educational Outcome 


Multiple R* 


N Students 


iV Centers 


Classroom Variables 


Remains on task 


.692 


927 


18 


Obeys rules 


.679 


493 


18 


Follows directions 


.666 


654 


17 


Completes homework 


.604 


799 


18 


Completes classwork 


.541 


640 


17 


Tardiness 


-.519 


212 


14 


Attends regularly 


.295 


238 


17 


Peer Relations Variables 


Relates appropriately 


.810 


489 


17 


Cooperates 


.705 


428 


17 


Participates 


.702 


281 


17 


Has friends 


.564 


222 


18 


Risk Variables 


Drop out risk 


-.662 


903 


18 


At risk for retention 


-.623 


472 


18 


At risk educationally 


-.555 


447 


17 


Global Variables 


Achieve above/below grade level 


-.570 


1020 


19 


Retained previously 


.475 


200 


17 


Academic proficiency (KERA) 


.401 


1770 


17 


Grades 


.336 


885 


18 



•All Significant at <.001 
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the predictor variable. In general, teacher active 
involvement with the center is the most consistent 
positive predictor of educational outcomes (13 out of 
18 significant beta wts. in the expected direction). 
Family and community involvement by centers, 
perhaps a proxy for the degree to which coordina- 
tors’ efforts are expended in this area, is often 
inversely related to educational outcomes (7 out of 18 
beta wts. inversely related to educational outcomes). 



The correlation between Overall Implementa- 
tion and the net change on academic proficiency 
was .401 (fK.OOl), indicating a significant positive 
relationship between extent and quality of imple- 
mentation and improvement on KERA scores. 
Where FRYSC program implementation was less 
strong, students tended to rise and fall in KERA 
categorization in equal numbers. Where program 
implementation quality was stronger, there was a 



Tables 

Relationship (beta coefficients) Between implementation Factors 
and Educational Outcomes 



Educational Outcome 


Teacher 

Knowledge/ 

Familiarity 


Teacher 

Active 

Involved 


Family & 
Community 
Involvement 


Program 

Mission 

Focus 


Teacher 
Awareness 
of Mission 


Classroom Variables 


Remains on task 


-.046 


.167*** 


-.115*** 


-.068* 


-.048 


Obeys rules 


.065 


.189*** 


-.122** 


-.003 


-.102* 


Follows directions 


.003 


.180*** 


-.070*** 


-.056 


-.040 


Completes homework 


-.090* 


.078* 


-.135*** 


-.086* 


-.041 


Completes classwork 


.017 


.114** 


-.112** 


-.018 


-.021 


Tardiness 


-.011 


.017 


.014 


.091 


-.159* 


Attends regularly 


-.015 


.056 


-.040 


.043 


.012 


Peer Relations Variables 


Relates appropriately 


-.016 


.204*** 


-.103* 


-.139** 


-.164*** 


Cooperates 


.095 


.269*** 


-.020 


-.068 


-.064 


Participates 


.144* 


.226*** 


.027 


-.055 


-.009 


Has friends 


.185* 


.035 


-.067 


.059 


-.035 


Risk Variables 


Drop out risk 


.093** 


-.161*** 


-.021 


.171*** 


.044 


At risk for retention 


.032 


-.171*** 


.109* 


.037 


.123** 


At risk educationally 


-.029 


-.220*** 


.001 


.000 


.034 


Global Variables 


Achieve above /below grade level 


.103*** 


-.124*** 


.075 


.012 


MS*** 


Retained previously 


-.018 


-.109 


.117 


.172 


.013 


Academic proficiency (KERA) 


.051* 


-.003 


-.008 


-.071 


.079*** 


Grades 


.019 


-.065* 


.069 


-.128 


.075* 



Significant at <.05*. <.01 **,<.001*** 
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net improvement of about 15% of students moving 
to a higher ranking. The number of students involved 
in this calculation was 1771. The factors that made 
the largest difference in accounting for this finding 
were Teacher Awareness of Center Mission, Program 
Mission Focus, and Teacher Knowledge/Familiarity 
With Center. 

Discussion 

The ICC map developed for this evaluation 
grew out of extensive involvement with centers and 
collaboration with coordinators who helped to 
develop it. It appears to have captured important 
components (“active ingredients”) of these innova- 
tive programs, as does the teacher survey which 
assesses “market penetration” in the schools. The 
results provide evidence in support of the premise 
of family support programs: that addressing the 
needs of at-risk children and their families may 
impact on students coming to school more prepared 
to learn, and thus enhance educational progress. It 
would appear that an important strategy for these 
programs is to establish close working relationships 
with educators and develop coordinated efforts to 
addressing students’ needs. The finding that 
community involvement is inversely related to 
educational outcomes may reflect the fact that 
putting one’s efforts into enhancing community 
supports and family connection with them, rather 
than spending more time in the school represents a 
long term strategy that may not show results as 
immediately as involvement with the school. 

As a field study, there are a number of limitations 
to this evaluation. One is the fact that these are 
rapidly expanding, full coverage programs that 
provide a wide variety of wraparound services to 
self-selected clients. This makes the securing of 
control conditions difficult and limits the evaluation 
to a correlational study. However, the strong findings 
of relationships between implementation and 



outcome variables is encouraging and suggests the 
possibility of replicating the essential elements of 
these programs under more controlled conditions. 
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